System for tuning parameters of a thermal sensor to improve object detection

ABSTRACT

Techniques associated with generating or tuning parameters associated with long wave infrared sensor data to improve object detection associated with the captured images are discussed herein. The system may determine a region of interest associated with the sensor data and adjust or tune the parameters to improve detection(s) within the region of interest. Additionally, the system may adjust the parameters based on map data and/or environmental conditions, such as weather and temperature.

BACKGROUND

Autonomous vehicles may use a variety of sensors to capture data fornavigating along routes. While navigating, the autonomous vehicles maydetect other objects in the environment and predict their behavior usingsensor data. Various different sensors may be used to detect objects indifferent operating conditions and environments.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different figures indicates similaror identical components or features.

FIG. 1 is an example block-diagram illustrating an example architectureof a parameter adjustment component associated with a sensor system, inaccordance with implementations of the disclosure.

FIG. 2 is another example block-diagram illustrating an examplearchitecture of a parameter adjustment component associated with asensor system, in accordance with implementations of the disclosure.

FIG. 3 is another example block-diagram illustrating an examplearchitecture of a parameter adjustment component associated with asensor system, in accordance with implementations of the disclosure.

FIG. 4 is an example pictorial view of long wave infrared (LWIR) sensordata with various parameter adjustments, in accordance withimplementations of the disclosure.

FIG. 5 is a flow diagram illustrating an example process associated withdetermining parameter adjustments for a LWIR sensor system, inaccordance with implementations of the disclosure.

FIG. 6 is another flow diagram illustrating an example processassociated with determining parameter adjustments for a LWIR sensorsystem, in accordance with implementations of the disclosure.

FIG. 7 depicts a block diagram of an example system for implementing thetechniques discussed herein.

DETAILED DESCRIPTION

Techniques for adjusting parameters for sensors to capture data of anenvironment are discussed herein. Sensors may be used by a vehicle, suchas an autonomous vehicle, to capture data to navigate to a destination.While navigating, the autonomous vehicle may encounter dynamic (e.g.,vehicles, pedestrians, animals, and the like) and/or static (e.g.,buildings, signage, parked vehicles, and the like) objects in theenvironment. In order to ensure the safety of the occupants and objects,the decisions and reactions of the autonomous vehicles to events andsituations that the vehicle may encounter, the vehicle may be equippedwith multiple types and instances of sensors to capture datarepresentative of the environment surrounding the vehicle. In somecases, the types of sensors may include thermal sensors (e.g., long waveinfrared (LWIR) sensors, mid-wave infrared (MRIR) sensors, short waveinfrared (SWIR) sensors, and the like) in conjunction with othersensors, such as lidar, visible-wavelength cameras (i.e., image capturedevices that capture data using visible wavelengths of light),microphones, short range infrared sensors, and the like.

Data from thermal sensor(s) may more clearly represent environments incertain environmental conditions (e.g., nighttime driving conditions,heavy rain, heavy snow, and the like) than other sensor types, such asvisible-wavelength cameras. As such, the thermal sensor data may be usedto assist with making navigational decisions and overall improve safeoperation of the autonomous vehicle. The techniques described herein aredirected to various aspects of improving the clarity and ability of theautonomous vehicle to detect objects using the thermal data, predictfuture behavior of the detected objects, and ultimately make safeoperational decisions based on the detected objects.

In some implementations, the autonomous vehicle, a parameter adjustmentcomponent, a thermal sensor module, a sensor module including an thermalsensor, and/or a combination of thermal sensor modules may be configuredto adjust parameters of the thermal sensor to improve object detectionwithin a desired region of the sensor's field of view. For instance, aparameter adjustment component may be configured to generate a histogramof values of pixels of an image captured by the thermal sensor and toselect one or more parameter adjustments for the thermal sensor toimprove object detection based on the histogram. In other instances, theparameter adjustment component may detect a region of interest withinthe image captured by the thermal sensor (e.g., an intersection, bikelane, turn lane, and the like) and to generate the histogram for thatregion of interest. Again, the parameter adjustment component may thendetermine parameter adjustments for the thermal sensor to improve objectdetection based on the histogram of the region of interest. In otherexamples, the parameter adjustment component may identify the region ofinterest from images captured by other sensors. For example, theparameter adjustment component may identify a region of low visibilitywithin a visible-light image and generate the histogram of the thermalimage corresponding to the region of interest. The parameter adjustmentcomponent may then determine parameter adjustments for the thermalsensor to improve object detection based on the histogram of the regionof interest.

In other implementations, the parameter adjustment component may beconfigured to determine areas within a thermal image that are outside anarea of interest or to provide overweight values to the histogram. Theparameter adjustment component may then remove the pixels associatedwith the overweight values from consideration when selecting parametersfor the thermal sensor. For example, if a large area of the sky iswithin a field of view of the thermal sensor, the parameter adjustmentcomponent may allow the parameters to be adjusted or tuned in a mannerin which clarity associated with pixels representing the sky is lost infavor of high clarity along the roadway or sidewalks. In some cases, thesky my be detected using visible-light image data and then acorresponding region of the sky in the thermal image may be excludedfrom the region of interest.

In some cases, the parameter adjustment component may also receive mapdata to assist with selecting or setting the parameters of the thermalsensor. For example, the map data may be usable to assist withdetermining a position of the vehicle (such as heading up or down hills)as well as to provide feedback related to particular location, such asunusually warm or cold areas (e.g., construction sites). In theseexamples, the parameter adjustment component may adjust one or moresettings (such as a gain or saturation) to compensate for the particularlocation. Similarly, the parameter adjustment component may determine ornarrow a region of interest when the map data indicates that the vehicleis driving up a hill to reduce the effect of the sky within the field ofview on the clarity of the reminder of the image data. In certain cases,the map data can be used to identify an object or area of interest(e.g., a relatively cold or hot object or area). The object or area ofinterest can be correlation to a portion of a field of view of a thermalsensor that may be excluded for purposes of thermal sensor parameteradjustment.

In some cases, the parameter adjustment component may also receiveenvironmental data to assist with selecting or setting the parameters ofthe thermal sensor. For example, if the autonomous vehicle is travelingin an environment experiencing heavy snow and/or rain, the parameteradjustment component may attempt to adjust one or more settings toimprove the object detectability within the thermal image data, as theheavy snow and/or rain may cause a drop in an expected temperature. Asanother example, the environmental data may indicate that the vehicle istraveling through or during an unusually hot day which may reduce theclarity of the thermal image data, particularly in environmentscontaining a large amount of roadways and buildings (such as in a cityenvironment). For instance, the environmental data may include a levelof urbanization for the surrounding environment. The parameteradjustment component may then adjust the parameter of the thermal sensorbased on the level of urbanization. As one illustrative example, thehigher the level of urbanization, the higher the expected temperaturerange. In this example, the parameter adjustment component maycompensate for the higher level of urbanization by increasing a dynamicrange or the thermal sensor.

In the examples above, a single thermal sensor and an image sensor arediscussed. However, it should be understood, that multiple thermalsensors as well as multiple other sensors (such as multiple visiblesensor) having a shared or overlapping field of view may be used tocapture the sensor data (e.g., the thermal data, the visible data, andthe like). It should also be understood, that the parameter adjustmentcomponent may be configured to provide multiple parameter adjustments toeach of the multiple thermal sensors sharing the field of view based onthe collected data. In some cases, the parameter adjustments for each ofthe thermal sensors may differ from each other. In this manner, thethermal sensor data from each thermal sensor may be tuned for differentor specific regions or interest or types of objects.

In various examples, the parameter adjustment component may beconfigured to adjust parameters comprising dynamic range, contrast,automatic gain correction, maximum gain, minimum temperature, maximumtemperature, tail rejection, region of interest, flat field correction,damping factors, smoothing factors and the like. For example, theparameter adjustment component may increase a minimum temperature of adynamic range in response to encountering a heavy snowstorm.

FIG. 1 is an example block-diagram illustrating an example architecture100 of a parameter adjustment component 102 associated with a sensorsystem 104, in accordance with implementations of the disclosure. In theillustrated example, a sensor system 104 or module may comprise at leastone thermal sensor as well as multiple other types of sensors (e.g.,image capture devices, lidar capture devices, infrared capture devices,motion sensors, microphones, and the like). Various parameters of thethermal sensor 104 may be tunable or otherwise adjustable to adjust theclarity of the captured sensor data 106 based on various environmentalfactors (such as time of day, temperature, weather conditions, field ofviews, surrounding landscape, and the like). For example, the thermalsensor 104 may be tuned or configured for a desired temperature or atemperature range. In cases in which an environmental temperature fallswell below and/or well above the desired temperature range, the sensordata may lose clarity, thereby increasing the difficulty associated withidentifying distinct objects within the resulting sensor data 106.

The architecture 100, discussed herein, can be configured to include theparameter adjustment component 102 to generate various parametersadjustments 108 and/or settings associated with the thermal sensorsystem 104 to improve object detection associated with the resultingsensor data 106. In the current example, the parameter adjustmentcomponent 102 may receive map data 110 associated with the environmentcurrently surrounding the autonomous vehicle. In some cases, the mapdata 110 may include elevation data, static object data (e.g.,buildings, trees, road signs, etc.), and the like. For example,parameter adjustment component 102 may be configured to determine theparameter adjustments 108 based on static object or landmark data thatcan be included in the map data 110. The static object data may includematerial properties, such as thermal proprieties, reflective properties,and the like. The landmark data can indicate a specific object or areathat may be detrimental to thermal imaging. A location of the landmarkor area can be associated with a corresponding portion of a field of aview of a thermal imaging sensor and precluded for use in adjustingparameter(s) of the thermal imaging sensor, as disclosed herein.Examples of landmark data can include a power plant, a refinery, thesun, building(s), bodies of water, etc.

The parameter adjustment component 102 may be configured to determinethe parameter adjustments 108 based on the map data 110. For example,the map data 110 may indicate that the autonomous vehicle is currentlytraveling through a city environment that includes large amounts ofconcrete, buildings, and other man-made structures that overall increasethe temperature of the environment, particularly in close proximity tothe roadways or drivable surfaces. In this example, the parameteradjustment component 102 may identify or determine an increase or deltain temperature caused by the city environment. The parameter adjustmentcomponent 102 may then adjust a gain setting, a range setting, and/or aminimum temperature setting to compensate for the increased temperatureassociated with the current environment.

As another illustrative example, the map data 110 may indicate that theautonomous vehicle is currently traveling up a steep incline. In thisexample, the thermal sensor system 104 may capture and the sensor data106 may comprise large amount of sky or otherwise empty space. In thesesituations, the thermal sensor system 104 may perform dynamicadjustments that attempt to improve clarity over the entire field ofview of the sensor 104. However, improving the clarity associated withthe sky or empty space, typically, does not impact object detectionand/or operational decisions of the autonomous vehicle. As such, in thecurrent example, the parameter adjustment component 102 may beconfigured to narrow a region of interest applicable to the dynamicadjustment system of the thermal sensor system 104. For instance, theparameter adjustment component 102 may, based at least in part on themap data 110 indicating the incline, cause the region of interest to bereduced to the lower three fourths, lower two thirds, lower half, lowerone third, and the like of the field of view of the thermal sensorsystem 104. In this manner, the thermal sensor 104 may, via a dynamicgain adjustment feature, tune the clarity only within the region ofinterest selected by the parameter adjustment component 102.

In some implementations, the parameter adjustment component 102 may alsoreceive environment data 112. For example, the environment data 112 maybe received from various sources, such as third-party weather reportingsystems, sensors on board the autonomous vehicle, as well as sensors onboard other nearby vehicles. The environment data 112 may include, as anon-exhaustive list representative list, a current temperature, anexpected temperature range, a time of day (e.g., dawn, morning, noon,late afternoon, night, etc.), weather conditions (e.g., rain, snow,hail, etc.), wind direction and/or speed, and the like. For example, theparameter adjustment component may narrow a region of interest, decreasea desired temperature range and the like when the autonomous vehicle istraveling through a snowstorm.

In the illustrated implementation, the parameter adjustment component102 may utilize recently captured sensor data 106 in adjusting theparameters for a subsequent interval of time. For example, the parameteradjustment component 102 may be configured to generate a histogramrepresentative of pixels associated with the sensor data 106 captured bythe thermal sensor system 104. The parameter adjustment component 102may then generate the parameter adjustments 108 based on the resultinghistogram. In some cases, the parameter adjustment component 102 mayfirst parse or analyze the sensor data 106 to determine a region ofinterest. For example, the sensor data 106 may represent a cross walk,intersection, bike lane, or the like that may be more likely to containdynamic objects. The parameter adjustment component 102 may thenidentify the high traffic area as a region of interest. The parameteradjustment component 102 may then generate a histogram over the regionof interest and generate the parameter adjustments 108 based on theresulting histogram for only the region of interest.

In this implementation, by incorporating the parameter adjustmentcomponent 102 into the architecture of the autonomous vehicle, theclarity of the sensor data 106 may be improved, at least over a regionof interest. The higher clarity sensor data 106 may then provide forfewer resources consumed by other systems 114 of the autonomous vehiclewith respect to object detection and tracking (such as association,segmentation, and the like).

In certain examples, the parameter adjustment component 102 may identifya region of interest to be excluded as a basis for parameter adjustmentfor a thermal sensor. For example, the sky may be a region of interestexcluded for parameter adjustment. Instead, the thermal sensor may betuned to better image a region of interest including the groundimmediately surrounding a vehicle. By excluding the sky, the thermalsensor may be better tuned to obtain information regarding the ground,for example. The horizon between the ground and the sky may bedetermined based on map data and/or incline data obtain from a sensor ofa vehicle, for example, and may be updated in real time. The horizon maybe determined by directly analyzing captured thermal sensor data. Incertain examples, the horizon can be augmented based on an environmentof the vehicle (e.g., if the vehicle is located in an urban environment,the horizon may be obscured or augmented by building(s). As discussedherein, additional regions of interest to the sky for exclusion can bedetermined using the disclosed techniques.

FIG. 2 is another example block-diagram illustrating an examplearchitecture 200 of a parameter adjustment component 202 associated withthermal sensor system 204, in accordance with implementations of thedisclosure. Similar to the architecture 100 discussed above with respectto FIG. 1, the parameter adjustment component 202 may be configured togenerate parameter adjustments 208 for the LWIR sensor system 204 basedon map data 210 and/or environment data 212.

In this example, the parameter adjustment system 202 may also receivevisible image data 216 from an image sensor system 218 in addition to orin lieu of the LWIR sensor data 208. In this example, the parameteradjustment component 202 may process, parse, or otherwise analyze thevisible image data 216 (which may be visible-light data) to determineparameters adjustment for the thermal sensor system 204. For instance,the parameter adjustment component 202 may determine areas representedwithin the visible image data 216 that are obstructed, unclear, dark,etc. that may result in difficulty for other systems 214 of theautonomous vehicle (such as perception and/or prediction system) todetect and track objects. As an illustrative example, the visible imagedata 216 may include one or more dark or shadowy areas (such as analley, covered walkway, and the like). It may be difficult for theperception and/or prediction system to identify objects and predictdynamic motion of the objects using the visible image data 216 withinthe shadowy areas.

In this example, the parameter adjustment system 202 may identify thedark or shadowy areas within the visible image data 216. The parameteradjustment system 202 may then focus a region of interest for thethermal sensor data 206 to correspond or otherwise align with the darkor shadowy areas within the visible image data 216. In general, thethermal sensor data 206 may more clearly represent objects (such aspedestrians, animals, and the like) despite a lack of visible lightdata. However, by adjusting the parameters of the thermal sensor systems204 to increase or improve object detectability of the thermal sensordata 206 at locations corresponding to the dark or shadowy areas, theparameter adjustment system 202 may further reduce the processing ordifficulty of the perception and/or prediction system in identifying andtracking objects within the dark or shadowy areas.

In the current example, the parameter adjustment system 202 is discussedwith respect to adjusting a region of interest using visible image data216. However, it should be understood that the parameter adjustmentsystem 202 may adjust any number of parameters associated with thethermal sensor system 204 to adjust resolution. Additionally, theparameter adjustment system 202 may be configured to tune parameters ofthe thermal sensor system 204 based on other types of sensor data, suchas lidar sensors, gyroscopes, temperature sensors, and the like.

FIG. 3 is another example block-diagram illustrating an examplearchitecture 300 of a parameter adjustment component 302 associated witha sensor system 304, in accordance with implementations of thedisclosure. Similar to the architecture 100 and 200 discussed above withrespect to FIGS. 1 and 2, the parameter adjustment component 302 may beconfigured to adjust parameters or settings associated with thermalsensor data 306 to improve the accuracy of and reduce the processingresources associated with object detection and tracking.

In the illustrated example, the parameter adjustment component 302 mayreceive the sensor data 306 as well as the map data 310 and/or theenvironment data 312 as discussed above. However, in this example, theparameter adjustment component 302 may adjust the sensor data 306 postcapture to improve the clarity for a particular region or based on themap data 310 and/or the environment data 312. For example, the parameteradjustment component 302 may apply gain settings to increase the clarityfor a particular region (e.g., along the roadway) at the expense ofclarity in other areas (e.g., the sky or empty space). In this case, theparameter adjustment component 302 may generate adjusted sensor data 316and provide the adjusted sensor data 316 to other systems 314 of theautonomous vehicle, such as perception and/or prediction system.

In the examples of FIGS. 1-3 various architectures are shown. It shouldbe understood that aspects of each architecture 100, 200, and/or 300 maybe used in combination with each other. For instance, the post sensordata processing discussed with respect to FIG. 3 may be used inconjunction with the parameter adjustment generation of FIGS. 1 and 2.

FIG. 4 is an example pictorial view 400 of thermal sensor data402(1)-(3) with various parameter adjustments, in accordance withimplementations of the disclosure. In the current example, the thermalsensor data 402(1)-(3) is shown with various parameters adjustments tosuccessively improve clarity along the roadway 404. For instance, in thecurrent example, the thermal sensor data 402(1) may include settingstuned to adjust for optimal clarity over the entire image. Thus, in thesensor data 402(1) the sky or upper region of the data 402(1) has someclarity but the important area along the roadway 404 has low contrastand may result in the perception and/or perception system havingdifficulty in identifying the preceding vehicle 406 as an object.

In the thermal sensor data 402(2) a parameter adjustment component (suchas parameter adjustment component 102, 202, and/or 302 of FIGS. 1-3) mayhave adjusted a parameter, such as a dynamic range parameter, contrastparameter, automatic gain correction parameter, maximum gain parameter,minimum temperature parameter, maximum temperature parameter, tailrejection parameter, damping parameter, smoothing parameter and thelike. As such, in the sensor data 402(2) the preceding vehicle 406 hasbecome more distinguished when compared with the background environmentthan in the thermal sensor data 402(1).

In the thermal sensor data 402(3) the parameter adjustment component mayhave selected a region of interest associated with the ground plane andadjusted the parameters to improve clarity along the ground at theexpensive of clarity within the sky. For instance, the sky has becomemostly dark and indistinguishable while the preceding vehicle 406 hasbecome clearer within the sensor data 402(3) with respect to the roadway404.

FIG. 5 is a flow diagram illustrating an example process 500 associatedwith determining parameter adjustments for a thermal sensor system, inaccordance with implementations of the disclosure. As discussed above,the thermal sensor data may more clearly represent surroundingenvironments than other sensors associated with an autonomous vehicle incertain environmental conditions, such as at night, in heavy rain orsnow, within fog, and the like. However, in some situations, the thermalsensor data may become saturated or otherwise lose clarity, such as whenthe thermal sensor system is exposed to large ranges in temperaturesacross the field of view. Therefore, a parameter adjustment componentmay be configured to adjust parameter sensors to improve clarity of thethermal sensor data during operation.

At 502, the parameter adjustment component of an autonomous vehicle mayreceive map data and/or environmental data associated with anenvironment proximate to an autonomous vehicle. The map data may includeelevation data, static object data (e.g., buildings, road signs, crosswalks, bike lanes, traffic lights, benches, and the like). Theenvironmental data may include weather conditions, temporaryobstructions (such as construction, detours, and the like), time of day,season, and the like.

At 504, the parameter adjustment component may receive sensor data froma thermal sensor system associated with the autonomous vehicle. Forexample, the sensor data may be associated with an interval of timeproceeding the current time.

At 506, the parameter adjustment component may determine a region ofinterest associated with the sensor data. For example, the parameteradjustment component may determine a region that has poor objectdetectability, an area of high traffic (such as a cross walk or bikelane, intersection, or other area likely to contain dynamic objects), orthe like, as a region of interest. In other cases, the parameteradjustment component may identify largely empty spaces (such as the sky,large buildings, empty fields, and the like) as areas to exclude fromthe region of interest.

As an illustrative example, when driving long a country road, largeportions of the field of view may include empty fenced farmland in whichit is highly unlikely to include a dynamic object capable of entering adrivable area. However, the thermal sensor system may reduce clarityassociated with the roadway in favor of increased clarity associatedwith the farmland. Thus, the parameter adjustment component may identifythe empty fenced farmland as an area to exclude from a region ofinterest 514 based on the sensor data and/or the map data. In thismanner, the parameter adjustment component may cause the thermal sensorsystem to automatically tune parameters for the high traffic areas(e.g., the roadway in this example), thereby improving clarity along theroadway at the expense of clarity within the fenced farmland.

In certain examples, an object or area within a field of view of avisual sensor can be used to identify objects or areas that would bedetrimental to thermal imaging for an autonomous vehicle. For example,the sun or a relatively hot or cold area/object (exceeding a temperaturethreshold) can be identified. Such areas/object may be detrimental toLWIR imaging by, for example, causing a dynamic range of a thermalsensor to include a relatively high or low temperature to be includedwithin the range. By excluding these areas/object, more steps can beassigned to a range more useful for autonomous driving (e.g., to gathermore information around grounded features in the vicinity of thevehicle).

At 508, the parameter adjustment component may generate a histogram ofpixel values associated with the region of interest 514. For example,the histogram may be utilized to represent an overall clarity associatedwith the region of interest 514, a temperature gradient associated withthe region of interest 514, an average temperature associated with theregion of interest, a maximum and/or minimum temperature associated withthe region of interest 514, a maximum and/or minimum clarity associatedwith the region of interest 514, and the like.

At 510, the parameter adjustment component may determine at least oneparameter adjustment for the thermal sensor system, based at least inpart on the map data, the environmental data, the histogram and/or acombination thereof. For instance, in various implementations, theparameter adjustment component may determine parameter adjustmentsassociated with a dynamic range, contrast, automatic gain correction,maximum gain, minimum temperature, maximum temperature, tail rejection,flat field correction, damping factors, smoothing factors and the like.

As an illustrative example, the parameter adjustment component mayincrease a minimum temperature of a dynamic range in response to theenvironmental data indicating the autonomous vehicle is travelingthrough a heavy snowstorm. In another illustrative example, theparameter adjustment component may adjust the dynamic range to improveclarity in regard to a smaller temperature gradient than expected withinthe region of interest 514.

At 512, the parameter adjustment component may send the parameteradjustment to the thermal sensor system. In some cases, the sensor datamay advance to a prediction and/or perception system of the autonomousvehicle, such that the autonomous vehicle may determine operationaldecisions based at least in part on the sensor data, and the parameteradjustments may be sent to the thermal sensor system. The thermal sensorsystem may then apply the parameter adjustments to the sensor datacaptured during the next interval.

FIG. 6 is another flow diagram illustrating an example process 600associated with determining parameter adjustments for a thermal sensorsystem, in accordance with implementations of the disclosure. In somecases, the thermal sensor system may be used in conjunction with and/orto supplement other sensor systems in capturing data for use inoperational decision making by an autonomous vehicle. In these cases, aparameter adjustment component associated with the thermal sensor systemmay be configured to determine parameter adjustments for the thermalsensor system based at least in part on the other sensor data, such asvisible image or sensor data.

At 602, the parameter adjustment component of an autonomous vehicle mayreceive map data and/or environmental data associated with anenvironment proximate to an autonomous vehicle. The map data may includeelevation data, static object data (e.g., buildings, road signs, crosswalks, bike lanes, traffic lights, benches, and the like). Theenvironmental data may include weather conditions, temporaryobstructions (such as construction, detours, and the like), time of day,season, and the like.

At 604, the parameter adjustment component may receive visible sensordata from a visible image sensor system associated with the autonomousvehicle. For instance, in some examples, the visible sensor data mayinclude areas that are obstructed, unclear, dark, shadowy, and the like.These obstructed, unclear, dark, and/or shadowy areas within the visiblesensor data may result in difficulties for other systems of theautonomous vehicles when making operational decisions. For instance, adark or shadowy region may contain a dynamic object (such as apedestrian) that is not detectable in the visible sensor data due to theshadow or darkness.

At 606, the parameter adjustment component may determine a region ofinterest associated with the thermal sensor system based at least inpart on the visible sensor data. For example, the parameter adjustmentcomponent may process, parse, or otherwise analyze the visible sensordata to determine an area, such as a region in the visible sensor datathat corresponds to areas that are obstructed, unclear, dark, shadowy,and the like. In some cases, the parameter adjustment component mayidentify areas within the visible sensor data that are both unclear ordark and likely to contain dynamic objects, such as an alleyway next tothe road, an overgrown area of a sidewalk, an exit to a fenced yard, andthe like. In some cases, the unclear or dark region may be identified asan area where objects are indistinguishable using the visible image dataor the boundary between objects is be defined within a threshold levelof certainty.

In this example, the parameter adjustment system may then set ordetermine a region of interest for the thermal sensor system tocorrespond or otherwise aligned with the area identified within thevisible sensor data. For instance, the sensor data generated by thethermal sensor system may more clearly represent different objects (suchas pedestrians, animals, and the like) despite the lack of light orclear visibility.

At 608, the parameter adjustment component may determine at least oneparameter adjustment for the thermal sensor system based at least inpart on the map data, the environmental data, the histogram and/or acombination thereof. For instance, in various implementations, theparameter adjustment component may determine parameter adjustmentsassociated with a dynamic range, contrast, automatic gain correction,maximum gain, minimum temperature, maximum temperature, tail rejection,flat field correction, damping factors, smoothing factors and the like.

At 610, the parameter adjustment component may send the parameteradjustment to the thermal sensor system. In some cases, the sensor datamay advance to a prediction and/or perception system of the autonomousvehicle, such that the autonomous vehicle may determine operationaldecisions based at least in part on the sensor data, and the parameteradjustments may be sent to the thermal sensor system. The thermal sensorsystem may then apply the parameter adjustments to the sensor datacaptured during the next interval.

FIG. 7 depicts a block diagram of an example system 700 for implementingthe techniques discussed herein. In at least one example, the system 700may include a vehicle 702, such the autonomous vehicles discussed above.The vehicle 702 may include computing device(s) 704, one or more sensorsystem(s) 706 (such as an thermal sensor system discussed above), one ormore emitter(s) 708, one or more communication connection(s) 710 (alsoreferred to as communication devices and/or modems), at least one directconnection 712 (e.g., for physically coupling with the vehicle 702 toexchange data and/or to provide power), and one or more drive system(s)714. The one or more sensor system(s) 706 may be configured to capturethe sensor data 716 associated with a surrounding physical environment.

In at least some examples, the sensor system(s) 706 may include thermalsensors (e.g., LWIR sensors), time-of-flight sensors, location sensors(e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurementunits (IMUs), accelerometers, magnetometers, gyroscopes, etc.), lidarsensors, radar sensors, sonar sensors, infrared sensors, cameras (e.g.,visible, RGB, IR, intensity, depth, etc.), microphone sensors,environmental sensors (e.g., temperature sensors, humidity sensors,light sensors, pressure sensors, etc.), ultrasonic transducers, wheelencoders, etc. In some examples, the sensor system(s) 706 may includemultiple instances of each type of sensors. For instance, time-of-flightsensors may include individual time-of-flight sensors located at thecorners, front, back, sides, and/or top of the vehicle 702. As anotherexample, camera sensors may include multiple cameras disposed at variouslocations about the exterior and/or interior of the vehicle 702. In somecases, the sensor system(s) 706 may provide input to the computingdevice(s) 704.

The vehicle 702 may also include one or more emitter(s) 708 for emittinglight and/or sound. The one or more emitter(s) 708 in this exampleinclude interior audio and visual emitters to communicate withpassengers of the vehicle 702. By way of example and not limitation,interior emitters can include speakers, lights, signs, display screens,touch screens, haptic emitters (e.g., vibration and/or force feedback),mechanical actuators (e.g., seatbelt tensioners, seat positioners,headrest positioners, etc.), and the like. The one or more emitter(s)708 in this example also include exterior emitters. By way of exampleand not limitation, the exterior emitters in this example include lightsto signal a direction of travel or other indicators of vehicle action(e.g., indicator lights, signs, light arrays, etc.), and one or moreaudio emitters (e.g., speakers, speaker arrays, horns, etc.) to audiblycommunicate with pedestrians or other nearby vehicles, one or more ofwhich may comprise acoustic beam steering technology.

The vehicle 702 can also include one or more communication connection(s)710 that enable communication between the vehicle 702 and one or moreother local or remote computing device(s) (e.g., a remote teleoperationscomputing device) or remote services. For instance, the communicationconnection(s) 710 can facilitate communication with other localcomputing device(s) on the vehicle 702 and/or the drive system(s) 714.Also, the communication connection(s) 710 may allow the vehicle 702 tocommunicate with other nearby computing device(s) (e.g., other nearbyvehicles, traffic signals, etc.).

The communications connection(s) 710 may include physical and/or logicalinterfaces for connecting the computing device(s) 704 to anothercomputing device or one or more external network(s) 734 (e.g., theInternet). For example, the communications connection(s) 710 can enableWi-Fi-based communication such as via frequencies defined by the IEEE802.11 standards, short range wireless frequencies such as Bluetooth,cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellitecommunication, dedicated short-range communications (DSRC), or anysuitable wired or wireless communications protocol that enables therespective computing device to interface with the other computingdevice(s). In at least some examples, the communication connection(s)710 may comprise the one or more modems as described in detail above.

In at least one example, the vehicle 702 may include one or more drivesystem(s) 714. In some examples, the vehicle 702 may have a single drivesystem 714. In at least one example, if the vehicle 702 has multipledrive systems 714, individual drive systems 714 may be positioned onopposite ends of the vehicle 702 (e.g., the front and the rear, etc.).In at least one example, the drive system(s) 714 can include one or moresensor system(s) 706 to detect conditions of the drive system(s) 714and/or the surroundings of the vehicle 702. By way of example and notlimitation, the sensor system(s) 706 can include one or more wheelencoders (e.g., rotary encoders) to sense rotation of the wheels of thedrive systems, inertial sensors (e.g., inertial measurement units,accelerometers, gyroscopes, magnetometers, etc.) to measure orientationand acceleration of the drive system, cameras or other image sensors,ultrasonic sensors to acoustically detect objects in the surroundings ofthe drive system, lidar sensors, radar sensors, etc. Some sensors, suchas the wheel encoders may be unique to the drive system(s) 714. In somecases, the sensor system(s) 706 on the drive system(s) 714 can overlapor supplement corresponding systems of the vehicle 702 (e.g., sensorsystem(s) 706).

The drive system(s) 714 can include many of the vehicle systems,including a high voltage battery, a motor to propel the vehicle, aninverter to convert direct current from the battery into alternatingcurrent for use by other vehicle systems, a steering system including asteering motor and steering rack (which can be electric), a brakingsystem including hydraulic or electric actuators, a suspension systemincluding hydraulic and/or pneumatic components, a stability controlsystem for distributing brake forces to mitigate loss of traction andmaintain control, an HVAC system, lighting (e.g., lighting such ashead/tail lights to illuminate an exterior surrounding of the vehicle),and one or more other systems (e.g., cooling system, safety systems,onboard charging system, other electrical components such as a DC/DCconverter, a high voltage junction, a high voltage cable, chargingsystem, charge port, etc.). Additionally, the drive system(s) 714 caninclude a drive system controller which may receive and preprocess datafrom the sensor system(s) 706 and to control operation of the variousvehicle systems. In some examples, the drive system controller caninclude one or more processor(s) and memory communicatively coupled withthe one or more processor(s). The memory can store one or more modulesto perform various functionalities of the drive system(s) 714.Furthermore, the drive system(s) 714 also include one or morecommunication connection(s) that enable communication by the respectivedrive system with one or more other local or remote computing device(s).

The computing device(s) 704 may include one or more processors 718 andone or more memories 720 communicatively coupled with the processor(s)718. In the illustrated example, the memory 720 of the computingdevice(s) 704 stores perception systems(s) 722, prediction systems(s)724, parameter adjustment systems(s) 726, as well as one or more systemcontroller(s) 728. The memory 720 may also store data such as sensordata 716 captured or collected by the one or more sensors systems 706,map data 730 and environment data 732. Though depicted as residing inthe memory 720 for illustrative purposes, it is contemplated that theperception systems(s) 722, prediction systems(s) 724, parameteradjustment systems(s) 726, as well as one or more system controller(s)728 may additionally, or alternatively, be accessible to the computingdevice(s) 704 (e.g., stored in a different component of vehicle 702and/or be accessible to the vehicle 702 (e.g., stored remotely).

The perception system 722 may be configured to perform object detection,segmentation, and/or classification on the sensor data 716, such as thethermal sensor data and/or the visible sensor data as discussed above.In some examples, the perception system 722 may generate processedperception data from the sensor data 716. The perception data mayindicate a presence of objects that are in physical proximity to thevehicle 702 and/or a classification or type of the objects (e.g., car,pedestrian, cyclist, building, tree, road surface, curb, sidewalk,unknown, etc.). In additional and/or alternative examples, theperception system 722 may generate or identify one or morecharacteristics associated with the objects and/or the physicalenvironment. In some examples, characteristics associated with theobjects may include, but are not limited to, an x-position, ay-position, a z-position, an orientation, a type (e.g., aclassification), a velocity, a size, a direction of travel, etc.Characteristics associated with the environment may include, but are notlimited to, a presence of another object, a time of day, a weathercondition, a geographic position, an indication of darkness/light, etc.For example, details of classification and/or segmentation associatedwith a perception system are discussed in U.S. application Ser. No.15/820,245, which are herein incorporated by reference in theirentirety.

The prediction system 724 may be configured to determine a trackcorresponding to an object identified by the perception system 722. Forexample, the prediction system 724 may be configured to predict avelocity, position, change in trajectory, or otherwise predict thedecisions and movement of the identified objects. For example, theprediction system 724 may include one or more machine learned modelsthat may, based on inputs such as object type or classification andobject characteristics, output predicted characteristics of the objectat one or more future points in time. For example, details ofpredictions systems are discussed in U.S. application Ser. Nos.16/246,208 and 16/420,050, which are herein incorporated by reference intheir entirety.

The parameter adjustment system 726 may be configured to determineadjustment parameters for the thermal sensor system based at least inpart on the sensor data 716, the map data 730, and/or the environmentdata 732. For instance, a parameter adjustment system 726 may beconfigured to generate a histogram of the values of the pixels of animage represented in the thermal sensor data and to select one or moreparameter adjustments for the thermal sensor to improve clarity based onthe histogram. In other instances, the parameter adjustment system 726may detect a region of interest within the image represented in thethermal sensor data and generate the histogram for that region ofinterest. Again, the parameter adjustment system 726 may determineparameter adjustments for the thermal sensor to improve clarity based onthe histogram of the region of interest. In other examples, theparameter adjustment system 726 may identify the region of interest fromsensor data 716 captured by other sensors. For example, the parameteradjustment system 726 may identify a region of low visibility or claritywithin a visible image data and generate the histogram of the thermaldata for a region of interest corresponding to the region of lowvisibility in the visible image data. The parameter adjustment system726 may then determine parameter adjustments for the thermal sensor toimprove clarity based on the histogram of the region of interest.

In other implementations, the parameter adjustment system 726 may beconfigured to determine areas within a thermal image that are outside anarea of interest. The parameter adjustment system 726 may then removethe pixels associated with the overweight values from consideration whenselecting the parameters for the thermal sensor system. For example, ifa large area of the sky is within the field of view of the thermalsensor system, the parameter adjustment system 726 may allow theparameters to be adjusted or tuned in a manner in which the clarityassociated with pixels representing the sky is lost in favor of highclarity along the roadway or sidewalks.

In some cases, the parameter adjustment system 726 may also receive mapdata 730 to assist with selecting or setting the parameters of thethermal sensor system. For example, the map data 730 may be usable toassist with determining a position of the vehicle 702 (such as headingup or down hills) as well as to provide feedback related to particularlocation, such as unusually warm or cold areas (e.g., constructionsites). In these examples, the parameter adjustment system 726 mayadjust one or more settings (such as a gain or saturation) to compensatefor the particular location. Similarly, the parameter adjustment system726 may determine or narrow a region of interest when the map data 730indicates that the vehicle 702 is driving up a hill to reduce the effectof the sky within the field of view on the clarity of the reminder ofthe sensor data 716.

In some cases, the parameter adjustment system 726 may also receiveenvironmental data 732 to assist with selecting or setting theparameters of the thermal sensor. For example, if the vehicle 702 istraveling in an environment experiencing heavy snow and/or rain, theparameter adjustment system 726 may attempt to adjust one or moresettings to improve the clarity of the thermal image data, as the heavysnow and/or rain may cause a drop in an expected temperature. As anotherexample, the environmental data 732 may indicate that the vehicle 702 istraveling through or during an unusually hot day which may reduce theclarity of the thermal image data, particularly in environmentscontaining a large amount of roadways and buildings (such as in a cityenvironment).

In at least one example, the computing device(s) 704 may store one ormore and/or system controllers 728, which may be configured to controlsteering, propulsion, braking, safety, emitters, communication, andother systems of the vehicle 702. The system controllers 728 maycommunicate with and/or control corresponding systems of the drivesystem(s) 714 and/or other components of the vehicle 702, which may beconfigured to operate in accordance with a route provided from aplanning system.

In some implementations, the vehicle 702 may connect to computingdevice(s) 736 via the network(s) 734. For example, the computingdevice(s) 736 may generate and provide the map data 730 and/or theenvironment data 732 to the vehicle 702. The computing device 736 mayinclude one or more processor(s) 738 and memory 740 communicativelycoupled with the one or more processor(s) 738. In at least one instance,the processor(s) 738 may be similar to the processor(s) 718 and thememory 740 may be similar to the memory 720. In the illustrated example,the memory 740 of the computing device(s) 736 stores the map data 730and the environment data 732. The memory 740 may also store a mapgeneration system 742 to assist with compiling and generating the mapdata 730.

The processor(s) 718 of the computing device(s) 704 and the processor(s)738 of the computing device(s) 736 may be any suitable processor capableof executing instructions to process data and perform operations asdescribed herein. By way of example and not limitation, the processor(s)718 and 738 can comprise one or more Central Processing Units (CPUs),Graphics Processing Units (GPUs), or any other device or portion of adevice that processes electronic data to transform that electronic datainto other electronic data that can be stored in registers and/ormemory. In some examples, integrated circuits (e.g., ASICs, etc.), gatearrays (e.g., FPGAs, etc.), and other hardware devices can also beconsidered processors in so far as they are configured to implementencoded instructions.

The memory 720 of the computing device(s) 704 and the memory 740 of thecomputing device(s) 736 are examples of non-transitory computer-readablemedia. The memory 720 and 740 can store an operating system and one ormore software applications, instructions, programs, and/or data toimplement the methods described herein and the functions attributed tothe various systems. In various implementations, the memory 720 and 740can be implemented using any suitable memory technology, such as staticrandom access memory (SRAM), synchronous dynamic RAM (SDRAM),nonvolatile/Flash-type memory, or any other type of memory capable ofstoring information. The architectures, systems, and individual elementsdescribed herein can include many other logical, programmatic, andphysical components, of which those shown in the accompanying figuresare merely examples that are related to the discussion herein. In someinstances, aspects of some or all of the components discussed herein caninclude any models, algorithms, and/or machine learning algorithms. Forexample, in some instances, the components in the memory 720 and 740 canbe implemented as a neural network.

CONCLUSION

While one or more examples of the techniques described herein have beendescribed, various alterations, additions, permutations and equivalentsthereof are included within the scope of the techniques describedherein. As can be understood, the components discussed herein aredescribed as divided for illustrative purposes. However, the operationsperformed by the various components can be combined or performed in anyother component. It should also be understood that components or stepsdiscussed with respect to one example or implementation may be used inconjunction with components or steps of other examples. For example, thecomponents and instructions of FIG. 7 may utilize the processes andflows of FIGS. 1-7.

A non-limiting list of objects may include obstacles in an environment,including but not limited to pedestrians, animals, cyclists, trucks,motorcycles, other vehicles, or the like. Such objects in theenvironment have a “geometric pose” (which may also be referred toherein as merely “pose”) comprising a location and/or orientation of theoverall object relative to a frame of reference. In some examples, posemay be indicative of a position of an object (e.g., pedestrian), anorientation of the object, or relative appendage positions of theobject. Geometric pose may be described in two-dimensions (e.g., usingan x-y coordinate system) or three-dimensions (e.g., using an x-y-z orpolar coordinate system), and may include an orientation (e.g., roll,pitch, and/or yaw) of the object. Some objects, such as pedestrians andanimals, also have what is referred to herein as “appearance pose.”Appearance pose comprises a shape and/or positioning of parts of a body(e.g., appendages, head, torso, eyes, hands, feet, etc.). As usedherein, the term “pose” refers to both the “geometric pose” of an objectrelative to a frame of reference and, in the case of pedestrians,animals, and other objects capable of changing shape and/or positioningof parts of a body, “appearance pose.” In some examples, the frame ofreference is described with reference to a two- or three-dimensionalcoordinate system or map that describes the location of objects relativeto a vehicle. However, in other examples, other frames of reference maybe used.

In the description of examples, reference is made to the accompanyingdrawings that form a part hereof, which show by way of illustrationspecific examples of the claimed subject matter. It is to be understoodthat other examples can be used and that changes or alterations, such asstructural changes, can be made. Such examples, changes or alterationsare not necessarily departures from the scope with respect to theintended claimed subject matter. While the steps herein may be presentedin a certain order, in some cases the ordering may be changed so thatcertain inputs are provided at different times or in a different orderwithout changing the function of the systems and methods described. Thedisclosed procedures could also be executed in different orders.Additionally, various computations that are herein need not be performedin the order disclosed, and other examples using alternative orderingsof the computations could be readily implemented. In addition to beingreordered, the computations could also be decomposed intosub-computations with the same results.

Example Clauses

A. An system comprising: one or more processors; and one or morenon-transitory computer-readable media storing instructions that, whenexecuted by the one or more processors, cause the one or more processorsto perform operations comprising: receiving visible sensor data from animage capture sensor associated with an autonomous vehicle; identifyingan area within the visible sensor data that contains a featuredetrimental to LWIR imaging for the autonomous vehicle; determining,based at least in part on the visible sensor data and the area, a firstregion of interest; determining a second region of interest associatedwith a long wave infrared (LWIR) sensor associated with the autonomousvehicle, the second region of interest corresponding to the first regionof interest; receiving LWIR sensor data from the LWIR sensor;determining, based at least in part on pixel values of LWIR sensor dataassociated the second region of interest, a histogram; determining,based on the histogram, a parameter adjustment for the LWIR sensor; andcausing the LWIR sensor to apply the parameter adjustment.

B. The system of claim A, wherein determining the first region ofinterest includes determining a region that is obstructed from view bythe visible sensor.

C. The system of claim A, wherein the area includes sky.

D. The system of claim A, wherein the feature corresponds to atemperature exceeding a threshold.

E. The system of claim A, wherein the operations further comprise:receiving map data and environmental data associated with an environmentproximate to the autonomous vehicle, and wherein determining theparameter adjustment is based at least in part on the map data and theenvironmental data.

F. A method comprising: receiving first sensor data from a thermalsensor associated with an autonomous vehicle; receiving second sensordata from a visible image sensor associated with the autonomous vehicle;determining, based at least in part on the second sensor data, a firstregion of interest; determining, based at least in part on the firstregion of interest, a second region of interest associated with thefirst sensor data; determining, based at least in part on the secondregion of interest, a parameter adjustment for the thermal sensor; andapplying the parameter adjustment to the thermal sensor or data obtainedfrom the thermal sensor.

G. The method of paragraph F, wherein the first region of interestassociated with the second sensor data comprises an obstructed region ofthe second sensor data.

H. The method of paragraph F, further comprising: determining, based atleast in part on the second region of interest, a histogram, wherein theparameter adjustment is based at least in part on the histogram.

I. The method of paragraph F, further comprising: receiving map dataassociated with an environment proximate the autonomous vehicle; anddetermining the parameter adjustment for the thermal sensor furtherbased at least in part on the map data.

J. The method of paragraph I, further comprising: receiving weather dataassociated with an environment proximate the autonomous vehicle; anddetermining the parameter adjustment for the thermal sensor furtherbased at least in part on the weather data.

K. The method of paragraph F, further comprising: receivingenvironmental data associated with an environment proximate theautonomous vehicle; and determining the parameter adjustment for thethermal sensor further based at least in part on the environmental data.

L. The method of paragraph F, wherein determining the first region ofinterest associated with the second sensor data further comprisesidentifying an empty region associated with the second sensor data andexcluding the empty region from the first region of interest.

M. The method of paragraph F, wherein the thermal sensor is a long waveinfrared (LWIR) sensor.

N. A non-transitory computer-readable medium storing instructions that,when executed, cause one or more processors to perform operationscomprising: receiving first sensor data from thermal sensor associatedwith an autonomous vehicle; receiving second sensor data from an imagesensor associated with the autonomous vehicle; determining, based atleast in part on the second sensor data, a first region of interest;determining, based at least in part on the first region of interest, asecond region of interest associated with the first sensor data;determining, based at least in part on the second region of interest, aparameter adjustment for the thermal sensor; and applying the parameteradjustment to the thermal sensor or data obtained from the thermalsensor.

O. The non-transitory computer-readable medium of paragraph N, whereinthe first region of interest associated with the second sensor datacomprises an obstructed region of the second sensor data.

P. The non-transitory computer-readable medium of paragraph N, whereinthe operations further comprise: determining, based at least in part onthe second region of interest, a histogram, wherein the parameteradjustment is based at least in part on the histogram.

Q. The non-transitory computer-readable medium of paragraph N, whereinthe operations further comprise: receiving map data associated with anenvironment proximate the autonomous vehicle; and determining theparameter adjustment for the thermal sensor further based at least inpart on the map data.

R. The non-transitory computer-readable medium of paragraph N, whereinthe operations further comprise: receiving environmental data associatedwith an environment proximate the autonomous vehicle; and determiningthe parameter adjustment for the thermal sensor further based at leastin part on the environmental data.

S. The non-transitory computer-readable medium of paragraph N, whereindetermining the first region of interest associated with the secondsensor data further comprises identifying an empty region associatedwith the second sensor data and excluding the empty region from thefirst region of interest.

T. The non-transitory computer-readable medium of paragraph N, whereinthe thermal sensor is a long wave infrared (LWIR) sensor.

U. A system comprising: one or more processors; and one or morenon-transitory computer-readable media storing instructions that, whenexecuted by the one or more processors, cause the one or more processorsto perform operations comprising: receiving long wave infrared (LWIR)sensor data from a LWIR sensor associated with an autonomous vehicle;receiving map data associated with an environment surrounding theautonomous vehicle; receiving environmental data associated with theenvironment surrounding the autonomous vehicle; determining, based atleast in part on the LWIR sensor data, the map data, and theenvironmental data, a region of interest associated with at least aportion of the LWIR sensor data; determining, based at least in part onpixel values of the LWIR sensor data associated with the region ofinterest, a parameter adjustment for the LWIR sensor; and causing theLWIR sensor to apply the parameter adjustment.

V. The system vehicle of claim U, wherein: the environmental dataincludes weather data; and determining the parameter adjustment for theLWIR sensor is based at least in part on a weather condition indicatedby the weather data.

W. The system vehicle of claim U, wherein the map data further compriseselevation data and the operations further comprise: receivingorientation data of the vehicle from an orientation sensor; and whereindetermining the region of interest is based at least in part on anelevation data and the orientation data.

X. The system vehicle of claim U, wherein the operations furthercomprise: receiving visible sensor data from a visible image sensorassociated with the autonomous vehicle, and wherein determining theregion of interest associated with at least the portion of the LWIRsensor data is based at least in part on the visible sensor data.

Y. The system vehicle of claim U, wherein the environmental dataincludes a characterization of environment surround the autonomousvehicle of a level of urbanization of the environment.

Z. A method comprising: receiving sensor data from a thermal sensorassociated with an autonomous vehicle; receiving map data or weatherdata associated with an environment proximate the autonomous vehicle;determining, based at least in part on the sensor data and at least oneof the map data or the weather data, a parameter adjustment for thethermal sensor based on; and applying the parameter adjustment to thethermal sensor or data from the thermal sensor.

AA. The method of paragraph Z, wherein the weather data indicates atemperature or environmental condition associated with the environmentand the parameter adjustment is based at least in part on thetemperature.

AB. The method of paragraph Z, wherein: the map data includes staticobject data; and determining the parameter adjustment is based at leastin part on a property of the static object.

AC. The method of paragraph Z, wherein: the map data includes staticobject data; and determining the parameter adjustment is based at leastin part on a relative location of the static object with respect to aposition of the autonomous vehicle.

AD. The method of paragraph Z, further comprising: determining, based atleast in part on the map data, a region of interest associated with thethermal sensor data; generating, based at least in part on the region ofinterest, a histogram associated with pixel values of the thermal sensordata; and wherein the parameter adjustment is based on the histogram.

AE. The method of paragraph AD, wherein determining the region ofinterest associated with the thermal sensor data further comprisesdetermining a region associated with an empty region and excluding theempty region from the region of interest.

AF. The method of paragraph AE, wherein the empty region represent sky.

AG. The method of paragraph AD, further comprising: receiving visibleimage data from an image sensor associated with the autonomous vehicle;and wherein determining the region of interest associated with the LWIRsensor data further comprises: determining an occluded region of thevisible image data; and identifying the region of interest as a portionof the LWIR sensor data that corresponds to the occluded region of thevisible image data.

AH. One or more non-transitory computer-readable media storinginstructions that, when executed, cause one or more processors toperform operations comprising: receiving sensor data from a thermalsensor associated with an autonomous vehicle; receiving map data orweather data associated with an environment proximate the autonomousvehicle; determining, based at least in part on the sensor data and atleast one of the map data or the weather data, a parameter adjustmentfor the thermal sensor based on; and applying the parameter adjustmentto the thermal sensor or data from the thermal sensor.

AI. The non-transitory computer-readable medium of paragraph AH, whereinthe weather data indicates a temperature associated with the environmentand the parameter adjustment is based at least in part on thetemperature.

AJ. The non-transitory computer-readable medium of paragraph AH, whereinthe weather data indicates an environmental condition and the parameteradjustment is based at least in part on the environmental condition.

AK. The non-transitory computer-readable medium of paragraph AH, furthercomprising determining a pose of the autonomous vehicle and wherein theparameter adjustment is based at least in part on the pose.

AL. The non-transitory computer-readable medium of paragraph AH, furthercomprising: determining, based at least in part on the map data, aregion of interest associated with the thermal sensor data; generating,based at least in part on the region of interest, a histogram associatedwith pixel values of the thermal sensor data; and wherein the parameteradjustment is based on the histogram.

AM. The non-transitory computer-readable medium of paragraph AL, whereindetermining the region of interest associated with the LWIR sensor datafurther comprises determining a region associated with an empty region,and excluding the region from the region of interest.

AN. The non-transitory computer-readable medium of paragraph AL, furthercomprising: receiving visible image data from an image sensor associatedwith the autonomous vehicle; and wherein determining the region ofinterest associated with the thermal sensor data further comprises:determining an occluded region of the visible image data; andidentifying the region of interest as a portion of the thermal sensordata that corresponds to the occluded region of the visible image data.

While the example clauses described above are described with respect toone particular implementation, it should be understood that, in thecontext of this document, the content of the example clauses can also beimplemented via a method, device, system, a computer-readable medium,and/or another implementation. Additionally, any of examples A-T may beimplemented alone or in combination with any other one or more of theexamples A-AN.

What is claimed is:
 1. A system comprising: one or more processors; andone or more non-transitory computer-readable media storing instructionsthat, when executed by the one or more processors, cause the one or moreprocessors to perform operations comprising: receiving long waveinfrared (LWIR) sensor data from a LWIR sensor associated with anautonomous vehicle; receiving map data associated with an environmentsurrounding the autonomous vehicle; receiving environmental dataassociated with the environment surrounding the autonomous vehicle;determining, based at least in part on the LWIR sensor data, the mapdata, and the environmental data, a region of interest associated withat least a portion of the LWIR sensor data; determining, based at leastin part on pixel values of the LWIR sensor data associated with theregion of interest, a parameter adjustment for the LWIR sensor; andcausing the LWIR sensor to apply the parameter adjustment.
 2. The systemas recited in claim 1, wherein: the environmental data includes weatherdata; and determining the parameter adjustment for the LWIR sensor isbased at least in part on a weather condition indicated by the weatherdata.
 3. The system as recited in claim 1, wherein the map data furthercomprises elevation data and the operations further comprise: receivingorientation data of the vehicle from an orientation sensor; and whereindetermining the region of interest is based at least in part on anelevation data and the orientation data.
 4. The system as recited inclaim 1, wherein the operations further comprise: receiving visiblesensor data from a visible image sensor associated with the autonomousvehicle, and wherein determining the region of interest associated withat least the portion of the LWIR sensor data is based at least in parton the visible sensor data.
 5. The system as recited in claim 1, whereinthe environmental data includes a characterization of environmentsurround the autonomous vehicle of a level of urbanization of theenvironment.
 6. A method comprising: receiving sensor data from athermal sensor associated with an autonomous vehicle; receiving map dataor weather data associated with an environment proximate the autonomousvehicle; determining, based at least in part on the sensor data and atleast one of the map data or the weather data, a parameter adjustmentfor the thermal sensor; and applying the parameter adjustment to thethermal sensor or data from the thermal sensor.
 7. The method as recitedin claim 6, wherein the weather data indicates a temperature orenvironmental condition associated with the environment and theparameter adjustment is based at least in part on the temperature. 8.The method as recited in claim 6, wherein: the map data includes staticobject data; and determining the parameter adjustment is based at leastin part on a property of the static object data.
 9. The method asrecited in claim 6, wherein: the map data includes static object data;and determining the parameter adjustment is based at least in part on arelative location of the static object data with respect to a positionof the autonomous vehicle.
 10. The method as recited in claim 6, furthercomprising: determining, based at least in part on the map data, aregion of interest associated with the sensor data; generating, based atleast in part on the region of interest, a histogram associated withpixel values of the thermal sensor data; and wherein the parameteradjustment is based on the histogram.
 11. The method as recited in claim10, wherein determining the region of interest associated with thesensor data further comprises determining a region associated with anempty region and excluding the empty region from the region of interest.12. The method as recited in claim 11, wherein the empty regionrepresents sky.
 13. The method as recited in claim 10, furthercomprising: receiving visible image data from an image sensor associatedwith the autonomous vehicle; and wherein determining the region ofinterest associated with the thermal sensor data further comprises:determining an occluded region of the visible image data; andidentifying the region of interest as a portion of the thermal sensordata that corresponds to the occluded region of the visible image data.14. One or more non-transitory computer-readable media storinginstructions that, when executed, cause one or more processors toperform operations comprising: receiving sensor data from a thermalsensor associated with an autonomous vehicle; receiving map data orweather data associated with an environment proximate the autonomousvehicle; determining, based at least in part on the sensor data and atleast one of the map data or the weather data, a parameter adjustmentfor the thermal sensor based on; and applying the parameter adjustmentto the thermal sensor or data from the thermal sensor.
 15. The one ormore non-transitory computer-readable media as recited in claim 14,wherein the weather data indicates a temperature associated with theenvironment and the parameter adjustment is based at least in part onthe temperature.
 16. The one or more non-transitory computer-readablemedia as recited in claim 14, wherein the weather data indicates anenvironmental condition and the parameter adjustment is based at leastin part on the environmental condition.
 17. The one or morenon-transitory computer-readable media as recited in claim 14, furthercomprising determining a pose of the autonomous vehicle and wherein theparameter adjustment is based at least in part on the pose.
 18. The oneor more non-transitory computer-readable media as recited in claim 14,further comprising: determining, based at least in part on the map data,a region of interest associated with the thermal sensor data;generating, based at least in part on the region of interest, ahistogram associated with pixel values of the thermal sensor data; andwherein the parameter adjustment is based on the histogram.
 19. The oneor more non-transitory computer-readable media as recited in claim 18,wherein determining the region of interest associated with the sensordata further comprises determining a region associated with an emptyregion, and excluding the region from the region of interest.
 20. Theone or more non-transitory computer-readable media as recited in claim18, further comprising: receiving visible image data from an imagesensor associated with the autonomous vehicle; and wherein determiningthe region of interest associated with the sensor data furthercomprises: determining an occluded region of the visible image data; andidentifying the region of interest as a portion of the thermal sensordata that corresponds to the occluded region of the visible image data.